-
Notifications
You must be signed in to change notification settings - Fork 489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
modern lstm #2752
modern lstm #2752
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2752 +/- ##
=======================================
Coverage 83.62% 83.62%
=======================================
Files 825 825
Lines 108686 108686
=======================================
Hits 90893 90893
Misses 17793 17793 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good! I am not so familiar with Rust and Burn as you. I am glad that you can revise the code. I'll learn from you on the design logic and formatting. |
Co-authored-by: Guillaume Lagrange <[email protected]>
Good! I am not so familiar with Rust and Burn as you. I am glad that you can revise the code. I'll learn from you on the design logic and formatting. |
The specific changes for the
The contribution process usually happens in a sequential manner. A user (you) submits code changes, a contributor (me) reviews the code and requests modifications where needed, and the user can provide his thoughts on the requests and make the necessary changes before asking for another round of review. And the process is typically repeated. But everything remains as part of the same pull request 🙂 In your last PR, the only changes required were structural (change how the example needed to be exposed in the workspace) so I made the changes myself since I had time. I'll let you make the changes, but if you need help let me know! |
It's weired that the model performance decreased a little after these changes. Is the latest burn still unstable or the crates in the I think I find the reason. I copied the code from the wgan-generate.rs (https://github.com/tracel-ai/burn/blob/main/examples/wgan/examples/wgan-generate.rs), where the autodiffbackend was used. But in modern-lstm inference, I think the backend without autodiff must be used, otherwise the predictions will be erroneous because of the layernorm and dropout. Now it works as expected when using backend without autodiff for inference. |
Oh good catch! The structure likely was carried over from another example, but I just fixed it in #2736. I'll fix the wgan example. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed some conflicts with the main branch.
Thanks for the great example 🙂 we might add this LSTM implementation to our actual modules in the future
Pull Request Template
Checklist
run-checks all
script has been executed.Related Issues/PRs
Provide links to relevant issues and dependent PRs.
Changes
Summarize the problem being addressed and your solution.
Testing
Describe how these changes have been tested.